{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 香浓公式\n",
    "$$\n",
    "S = -P(x)\\log P(x)\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 豹子的信息\n",
    "$$\n",
    "\\begin{aligned}\n",
    "P(666) &= \\frac{1}{6^3} \\\\\n",
    "S(666) &= -P(666)\\log P(666) \\\\\n",
    "&=- \\frac{-\\log 6^3}{6^3} \\\\\n",
    "&= \\frac{3\\log 6}{6^3}\n",
    "\\end{aligned}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# ID3\n",
    "$$\n",
    "Power(x_i) = S(X) - S(x_i) \n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "| 财富 | 相貌 | 身高 | 行动 |\n",
    "| ---- | ---- | ---- | ---- |\n",
    "| 富   | 帅   | 高   | 追   |\n",
    "| 富   | 帅   | 矮   | 追   |\n",
    "| 富   | 丑   | 高   | 不追 |\n",
    "| 富   | 丑   | 矮   | 不追 |\n",
    "| 穷   | 帅   | 高   | 追   |\n",
    "| 穷   | 帅   | 矮   | 不追 |\n",
    "| 穷   | 丑   | 高   | 不追 |\n",
    "| 穷   | 丑   | 矮   | 不追 |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\begin{aligned}\n",
    "S(行动) &= S(追) + S(不追) \\\\\n",
    "&= - \\frac{3}{8} \\log \\frac{3}{8} - \\frac{5}{8} \\log\\frac{5}{8} \\\\\n",
    "&= \\frac{3 \\log \\frac{8}{3} + 5 \\log \\frac{8}{5}}{8} \\\\\n",
    "&= \\frac{8\\log8 - 3\\log3 - 5\\log 5 }{8} \\\\\n",
    "&= \\frac{1}{8}\\log \\frac{8^8}{3^3 * 5^5}\n",
    "\\end{aligned}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\begin{aligned}\n",
    "S(行动|财富) &= S(行动|富) + S(行动|穷) \\\\\n",
    "&= S(追|富) + S(不追|富) + S(追|穷) + S(不追|穷) \\\\\n",
    "&= - \\frac{2}{4}\\log\\frac{2}{4} - \\frac{2}{4}\\log\\frac{2}{4} - \\frac{1}{4}\\log\\frac{1}{4} - \\frac{3}{4}\\log\\frac{3}{4} \\\\\n",
    "&= -\\frac{4}{4}\\log\\frac{2}{4} - \\frac{1}{4}\\log\\frac{1}{4} - \\frac{3}{4}\\log\\frac{3}{4} \\\\\n",
    "&= \\frac{1}{4}(4 \\log \\frac{4}{2} + \\log \\frac{4}{1} + 3\\log \\frac{4}{3}) \\\\\n",
    "&= \\frac{1}{4}(8\\log4 - 4\\log2 - 3\\log3) \\\\\n",
    "&= \\frac{1}{4}(8\\log4 - 2\\log4 - 3\\log3) \\\\\n",
    "&= \\frac{1}{4}(6\\log4 - 3\\log3) \\\\\n",
    "&= \\frac{1}{4}\\log\\frac{4^6}{3^3}\n",
    "\\end{aligned}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\begin{aligned}\n",
    "S(行动|相貌) &= S(追|帅) + S(不追|帅) + S(追|丑) + S(不追|丑) \\\\\n",
    "&= - \\frac{3}{4}\\log\\frac{3}{4} - \\frac{1}{4} \\log \\frac{1}{4} - 0 - 1\\log1 \\\\\n",
    "&= \\frac{1}{4}(4\\log4 - 3\\log3) \\\\\n",
    "&= \\frac{1}{4} \\log\\frac{4^4}{3^3}\n",
    "\\end{aligned}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\begin{aligned}\n",
    "S(行动|身高) &= S(追|高) + S(追|矮) + S(不追|高) + S(不追|矮) \\\\\n",
    "&= -\\frac{2}{4}\\log\\frac{2}{4} - \\frac{1}{4}\\log\\frac{1}{4} - \\frac{2}{4}\\log\\frac{2}{4} - \\frac{3}{4}\\log\\frac{3}{4} \\\\\n",
    "&= -\\frac{4}{4}\\log \\frac{2}{4} - \\frac{1}{4}\\log\\frac{1}{4} - \\frac{3}{4}\\log \\frac{3}{4} \\\\\n",
    "&= \\frac{1}{4}\\log\\frac{4^6}{3^3}\n",
    "\\end{aligned}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\left\\{\n",
    "\\begin{matrix}\n",
    "Gain(行动，财富) = S(行动) - S(行动|财富) \\\\\n",
    "Gain(行动，相貌) = S(行动) - S(行动|相貌) \\\\\n",
    "Gain(行动，身高) = S(行动) - S(行动|身高)\n",
    "\\end{matrix}\n",
    "\\right.\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们优先选择的是那些能够获得更多信息增益的标准，综合衡量下来，长相还真尼玛重要。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
